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/\n equation was derived to detcniunc the rclationsliip between the 
pooled witiiin-subproiq^ r and tlie r obtnined frora tlie total proup data. 
It was, tiros, possible to assess t]ie ar.ount of distortion introduced by 
pooling; heterogeneous subgroups. As a basis for deciding whctlicr to 
pool two subgroups in order to calculate a single r for the total group, 
a two- stage procedure was reconnended: (1) comparison of the two 
witiiin- subgroup r*s; and (2) comparison of the total group r and the 
peeled witidn- subgroup r. On the basis of results for the second stage 
test, distortion in the total group r was shown to be a function of 
tlie pattern of subgrouj^ mean differences, total group sample size, and 
the iMgnitude of the pooled within- subgroup r. Implications were dis- 
cussed. 



'"'Paper presented at the annual meeting of tiie >\ricrican Lducational 
Research Association, Chicago^ 1974. Portions of this paper will be 
publislied in tiic Suruuer, 1975 issue of Lducational and P sychological 
Nteasurement . 
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r*rc(iuently , in tlie psycliolopical and educational literature, corre- 
lational studies are reporter in wliich product -norxMit correlations are 
calculated beLv.ecn two variables for sets of data pooled across two, 
possibly lietero^^,eneous, categorical subgroups. Tlic pooling', of hetero- 
geneous subgrouj^s to calculate a product -moiuent correlation was first 
discussed by Karl Tearson (Pearson, Lee, 5 lJramley-^kx)re, 1899). hy way 
of illustration, tliese authors presented correlational data betvveen 
length and breadth of skulls for 806 males (r = .0869) and 340 females 
(r « -.0424). When the two subgroups were pooled, an r of .1968 was 
obtained, and tliis jr was considered to represent a large spurious corre- 
lation. 

Fron a saripling of recent introductory statistics textbooks in 
psychology and education, it was found that writers do discuss the 
effects on the correlation coefficient resulting from pooling hetero- 
geneous subgroups (Games § Klare, 1967; Glass 5 Stanley, 1970; Guilford, 
1965; U'alker 5 Lev, 1969; among others). V\here references are made, 
Dunlap's (1937) paper on the combinative properties of correlation 
coefficients is most frequently cited. Dunlap presented a method for 
calculating a total group correlation coefficient from subgroup corre- 
lation coefficients and the means and standard deviations of the vari- 
ables . 

To date, a matliematical formulation of the effects on the corre- 
lation coefficient from pooling heterogeneous subgroups has been lacking. 
In lieu of such a formulation, textbook writers have tended to stress 
cautious interpretation and the use of subgroup correlation coeffi- 
cients to lielp provide a rational explanation for correlation results 
in the total groip. Tlie major interest of this paper is the derivation 
of a mathematical formulation and the demonstration of the effects of 



pooling two subgroups on the total group correlation coefficient. Of 
additional interest is a procedure to guide the decision concerning 
the pooling of data for the purj^ose of calculating a single correlation 
coefficient. 

Fonnulation 

Given two subgroups, let n^ and n^ ^'^^ subgrouj) sanijile sizes, 
where n * Let U and V be the distances Ixtv/een the subgroup 
means for X and Y, respectively, i-^ - , U = ^ - and V = ^2 ' II* 
S un of S(|uires and Cross Products 

llie suKi of cross products for the total group, SS(XY)^, can be 
defined 

SS(XY)^ = SS(Xi')^ ^ SSm)2 ^ |-). W 
where SS (XY) and SS(XY)^ arc the within- subgroup sums of cross 
products. Since tiie sum of squares for X in the total group is actu- 
ally tlie sum of cross products with respect to itself, 

SS(X)^ - SS(X)^ ^ SSCX)^ ^ H^)^ [2] 

Similarly, for Y, 

SS(Y)^^ « SS(Y)^ ^ SS(Y)2 ^ P] 
Total Group r^. 

Using liquations 1, 2, and 3, the correlation coefficient for the 
total group is: 

SS(XY)j ♦ SS(XY), ♦ lJy(n^n2/N) 



Jt " 



/{SS(X)^ ♦ SS(X)2 ♦ U2(n^n2/N)}{SS(Y)^ ♦ 5S(Y)2 * Y2(n^n2/N)}' 
An equivalent form of this equation was derived by Dunlap (1937). If 
§S(XY)^ = SS(XY)^ ♦ ^®2, SS(X)^^. = SS(X)^ ♦ SS(X:^, and SSCY)^ » 
SSCY)^ ♦ ^(l)^' equation dcfininj^ may be simplified: 
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« — • ^ [4] 

llie correlation coefficient for the total group is therefore expressed 

in terns of pooled \>ithin-subf;roui> sunis of squares and cross products . 

subgroup saq)le sizes, and distances between the subgroup means. 

Pooled Within-Subprouj") r^^. 

Tlie pooled within- subj.\roup correlation coefficient is obtained 

from pooling of tlic subgrouji suns of squares and cross products: 

SS(XY), ♦ SSCXy)^ SS(XY), 
r = i ^ ^ b: rjT 

^ •^(SSCX)! * SSCX)2}{SSCY)i > SSQO2) /SSQO^SS(Y)^' 
It should be clear from inspection of Equation 5 that for equal vari- 
ances of X in both subgroups and equal variances of Y in both subgroups, 
r^^ is a weighted ariUimetic mean of the within- subgroup correlations, 
weighted by tJie number of observations in eacli subgroup. 

Furthermore, may be compared to r^ in two ways. First, r is 
a special case of resulting when subgroup mean differences are non- 
existent (i.e. . Us V » 0). Second, is that special case of r^ when 
subgroup differences are eliminated statistically. The latter compari- 
son requires the form of a first-order partial correlation I^^^fZ' 
where and are the two variables measured in tlie total group and 
^ is a dichotomous variable indicating subgroi^i membership. In the 
formula for the first-order partial correlation, since r^ . and r„ . . 
are point-biserial correlations, the result of operating i^on the 
formula for tl\c first-order partial correlation is: 

SS(XY)j - Uy(nin2/N) 



r * _ , 

* - /{SSQC)^ - ii2Cnin2/N)}{SSCY)^ - V2 (nin2/N) ) 

Since the above ctiuation re[)rcsents an alternative definition of r^, 
then r., the pooled within-suhgroup correlation coefficient, is also 
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the result of statistically eliminatinp. subgroup differences from the 
total group correlation coefficient, llie latter definition of r^^ sug- 
gests that r^^. can be meaningfully used as a descriptive statistic with 
a kno\ai sampling distribution. 
Furth er Derivation of 

If tlie numerator and denominator of Lquation 4 arc eacli divided by 
the product of the pooled within- subgroujT standard errors of the mean 
(Sjr and Sj^,- ), a more coi;venicnt definition of r^ arises. To complete 
this series of operations, by defining t^ = ^^^^^ " -^-Xyj 

subgrouj^ mean differences measured in units of standard errors, the 
following final form results: 

(N - 2)r,^. t^t, 

= . - " . [6J 

^ /{(N - 2) ♦ t2}{(N - 2) ♦ t2} 

Ik JL 

In this form, r^ is defined in terms of total group Scur^lc size, the 
pooled within- subgroup correlation coefficient, and subgroup mean 
differences that are measured in units distributed as Student's ^ under 
the appropriate assuimtions. 

The Decision to Pool 

The following is a simple two-stage procedure, recommended as a 
basis for the decision to pool two subgroups in order to calculate a 
single correlation coefficient for the total group data. 

1. T\\c two within- subgrouj) correlation coefficients should be 
compared. Under the h>'pothcsis H^: pj^ « unit-normal £ test, 

relying on Fisher's r-£ transformation (££)# can be used: 



2. ' ==:. 
/ 1 * 1 

If U_ is rejected, it is unreasonable to calculate as a measure of 
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correlation for the' two subj; roups. If is accepted, tlicn r^^. can bo 
considered a useful measure of correlation for the two subproups, and 
the second stage sliould be followed. 

2\ In order to assess the distortion introduced by pooling the 
t^vo sul/groups, should be conpared to r^^. under the hypothesis fj^: 
^t " ^^^^ test, if rj^. is considered an estimate of py, 

Zf - 2f 



If is rejected, then r^ may be considered distorted, but r^^, can be 
used as a measure of correlation for the two subgroups. If is 
accepted, then pooling the data from the two subgroups in order to 
calculate a total groi^ correlation coefficient appears to be a parsi- 
monious and reasonable procedure. 



Examples of Distor^i'^n 



According to liquation 6, distortion in r^ is affected by subgroup 
centroid differences and total group sair^^le size. Three patterns of 
subgroup centroid differences are of interest and are shown in Figure 1. 



Insert Figure 1 about here 



(1) If the mean of Subgroup 2 is higher than the mean of Subgroup 1 on 
both variables (Cases a, b , and c of Figure 1) , tlie greater the dif- 
ference between the subgroups on the two variables, the more exaggerated 
the value of r^ in a positive direction. (2) If tlie mean of Subgroup 2 
is higher than the mean of Subgroup 1 on one variable, and equal on the 
other variable (Cases d, e , and f of Figure 1) , the greater the dif- 
ference between the two subgroups on tlie one variable, the closer r^ 
is to a value of 0.00. (3) If the mean of Subgroup 2 is higher than 
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the rncaji of Subgroui^ 1 on one variable, and lower on the other variable 
(Cases £, h, and i, of Fipurc 1), the greater the difference bctv.een 
tlie subgroui^s on the two variables, the more cxaeRcratcd tl\e value of 
r^ in a negative direction. Furthermore, for constant differences 
betvv'ecn the subf;rouj^ ccntroids as nveasurcd in standard errors, increas- 
ing the total group sample size serves to minimize tlie effects of sub- 
group centroid differences, i.e., r^ approaches r^^. 

Tlie effects of subgroup sajnple size discrepancy on the calculation 
of r^. and r^^ can be shown by reference to Equations 4 and 5. According 
tc bciuation 5, for constant total group sample size, the larger the 
discrepancy between the subgroi^ sample sizes, tJie greater the influence 
of the larger subgroup in the calculation of r^^. In addition, according 
to Equation 4, for constant differences between subgroup centroids and 
for constant total group sample size, the larger the discrepancy between 
nj^ and n2f the smaller the effect of subgroip centroid differences in 
the calculation of r^, and, thus, the more equal the values of and r^. 

For the second stage test, in order to demonstrate tJie anount of 
distortion introduced by pooling heterogeneous subgroups. Equation 6 
was utilized to calculate r^ under varying sanple conditions. The 
sanple conditions were derived from contoinations of four magnitudes of 
subgroup centroid differences for the three patterns, five total group 
saiiple sizes (N), and three values of r^. By assuming two biv te 
normal populations, the magnitude of difference between subgroup means 
can be represented by enploying four-decimal critical values of Student's 
t distribution for 2^<.05, ]i<.01, jk.001, and 2<.0001, obtained for N-2 
df from Sockloff § Edney's (1972) tables. The .001 and .0001 signifi- 
cance levels were used because these levels represent extreme differences 
that are sometimes found in researcli data, altliough not necessarily 

O 
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reported, Hie five total group sample sizes were 10, 50, 100, 200, and 
1000, 'Hie three values of r^^. were ,8000, .4000, and 0.0000, chosen to 
represent liijih, moderate, and low correlations, respectively. 

Tables 1, 2, and 3 present calculated values of r^^ derived from 
the values of r^^, under the varyinr, samj^le conditions. Also included 
in these tables arc tlie second stage tvvo-tailed tests of U^: « 
to assess the amount of distortion introduced under the ccnditions. 
Negative values of r^^. are not shovax in the tables since the effects of 
the three patterns for negative r^^'s are opposite in sign from those 
sho\vn for positive r^^'s, e.g., the amount of exaggeration in a positive 
direction for a positive correlation under Pattern 1 is equal to the 
amount of exaggeration in a negative direction for a negative corre- 
lation under Pattern 3. 

As shouTi in Table 1, under Patterns 1 and 2, large differences 

Insert Table 1 about here 

between the subgroup means have a small effect on the value of r^^when 
I- » .8000. Under both patterns, a total group sample size of 50 
appears to be sufficient to minimize the distortion introduced by 
subgroup mean differences that are significant at the .0001 level. On 
the other liand, the results were quite different under Pattern 3. When 
the means of die two subgroups are significantly different at the .0001 
level, but in opposite directions, for a total group sample size of 50 

was calculated to be .3089, which is significantly different from 
an of .8000. Furthermore, even for a total group sample size of 
1000, tlie pooling of subgroups when subgroup means differ in opposite 
directions at tlie .0001 level produced an of .7729. Although this 
value of x\ is significantly r I'* ^erent from an r, at the .05 level. 
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one call arpue that such statistically significant differences between 
Lt ly '^^^^ little practical significance. 

According to Table 2, wfien r^^ ° .4000, the results for Pattern 1 

Insert Table 2 about here 

suggest that total group sample sizes of 50 are sufficient to avoid 
distortions introduced by pooling subgroups when both sets of subgroup 
means differ significantly in tlie same direction at tJie ,0001 level. 
Under Pattern 2, significant distortion was not found, even for a total 
group sainple size of 10. The Pattern 3 results suggest that a total 
group sample size of 200 will avoid distortion in the calculation of r^. 
According to Table 3, total group sample sizes of 50 appear to be 

Insert Table 3 about here 

sufficient to avoid distortion when r , ■ 0.0000 and the two sets of 
subgroup means differ at the .0001 level. Based on the symmetry of the 
sampling distributions of r when p ■ 0, this conclusion holds for subgroup 
means differing in the same or opposite directions. 

Discussion 

The various results clearly suggest the varieties of distortion 
that may be introduced by haphazardly pooling subgroups of data for the 
purpose of calculating a single correlation coefficient. The two*stage 
test procedure should offer protection against such distortions. In 
addition, it was shown that greater latitude exists in terms of non* 
distorting pooling when the subgroup) mean differences are small, the 
subgroup sample sizes are large » and the pooled within* subgroup corre- 
lations are low to moderate. The calculated examples suggest limits 
within whidi distortion does not seriously affect correlational results. 
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llic types of subf^roups to which this discussion refers arc those 
result inj; from natural dichotomies and tliosc resulting fror.i an arbitrary 
split where (a) the decision to split the total group was based on 
considerations other tiian that of ridding the data of non-linearity, and 
(b) middle range data has been discarded. Uhcn an arbitrary split is 
made to rid the total group data of non-linearity, the two subgroups 
may show evidence of different, but linear, relationships. According 
to the first stage test, if the two within- subgroup correlations arc 
different, then it would appear unreasonable to even consider pooling 
the data on the basis of the original rationale for having mrdc the 
split. Oil the other hand, if middle range data is not discarded when 
an arbitrary split is made for reasons other than ridding tlie data of 
non-linearity, then pooling uT)uld appear to be a reasonable step 
toward restoring the information contained within the total bivariate 
set of data. 

Implications of tliis study relate to the use of the pooled wi thin- 
subgroup correlation coefficient, to current practices in educational 
research, and to generalizations of this study in terms of pooling 
multiple su})groups in the calculation of correlation matrices. First, 
assuming no difference between the within- subgroup correlation coeffi- 
cients (non- rejection of the first stage test), the pooled within- subgroup 
correlation coefficient is useful as a descriptive statistic with 
hypothesis- testing capabilities resulting from its equivalence to a 
first-order partial correlation coefficient. Second, in research 
involving multiple dependent measures that are analyzed via several 
ANOVA's, rather than MANOVA, intercor relations among the dependent 
measures should be assessed through pooled within-subgroup correlation 
coefficients rather than total group correlation coefficients. Otherwise, 

ERLC 
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the n)c?nnini: fulness of the intcrcorrclatlons would be continpent tipon 
tlic failure to find subgroui> differences in all of tlie ANWA's. 

Last, consider iuR the dcnonstrated varieties of possible distortion 
of a single correlation coefficient from tlie harhazard poolinp of only 
two hctcroj;encous subgroups, the generalizations of these results must 
be inliercntly r.orc complex, i.o», the effects of pooling multiple sub- 
groups on correlation matrices. If, indeed, such complex distortion 
can be dcnonstrated, and it is desirable to pool data for reasons of 
parsimony, this suggests that further study should be devoted to the 
multivariate case and tJie development of appropriate test procedures. 
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Tabic 1 

Values of RcsultinR from Hircc Patterns of 
Subgroup Ccntroid Differences and I'ive Total 
Group Sample Sizes: r, ■ .8000 



Total group 
sanf)lc size 


Significance levels for t_ distribution when 
subgroup mean differences equal critical values 


I)<.05 


ll<.oi 


j><.001 


£<.0001 



Pattern 1: ^ > ^, ^2 > Yj, both significant 



10 


.8799 


.9169 


.9521 


.9727* 


50 


.8155 


.8261 


.8408 


.8546 


100 


.8077 


.8132 


.8210 


.8288 


200 


.8039 


.8066 


.8107 


.8148 


1000 


.8008 


.8013 


.8022 


.8030 


Pattern 2: ^2 > 


T^, significant; 7^*72 


10 


.6200 


.5156 


.3914 


.2953 


50 


.7683 


.7460 


.7138 


.6822 


100 


.7844 


.7732 


.7568 


.7403 


200 


.7923 


.7867 


.7784 


.7699 


1000 


.7985 


.7973 


.7957 


.7940 


Pattern 3: ^2 * 1 


j^, 7^ < T^, both significant 


10 


.0813* 


-.2523*** 


-.5691**** 


-.7547**** 


50 


.6602* 


.5654** 


.4332**** 


.3089**** 


100 


.7305 


.6816** 


.6108*** 


.5412**** 


200 


.7653 


.7405* 


.7040** 


.6672**** 


1000 


.7931 


.7881 


.7806 


.7729* 



Note. --Asterisks refer to significance levels of unit-normal 
£ tests conparing r^ and i^. 
• £<.05 

£<.01 

£<.001 
•••• £<.0001 




Table 2 

Values of Resulting: from Ihrce Patterns of 
Subgroup Centroid Differences and Five Total 
Group Sample Sizes: r. = .4000 



Total group 
sample size 


Significance levels for t^ distribution when 
subgroup mean differences equal critical values 


Ii<.05 


ji<.01 


£<.001 


£<.0001 


Pattern 1: ^2 * Z 




bol significant 


10 


.6396 


.7508 


.8564 


.9182** 


50 


.4466 


.4782 


.5223 


.5637 


100 


.4232 


.4395 


.4631 


.4863 


200 


.4116 


.4198 


.4320 


.4443 


1000 


.4023 


.4040 


.4065 


.4090 


Pattern 2: > 


T^t significant; ' ^ 


•2 


10 


.3100 


.2578 


.1957 


.1477 


50 


.3842 


.3730 


.3569 


.3411 


100 


.3922 


.3866 


.3784 


.3701 


200 


.3961 


.3933 


.3892 


.3850 


1000 


.3992 


.3987 


.3978 


.3970 


Pattern 


3: T^>1^ 


[* T2 ^ Ti* significant 


10 


-.1590 


-.4184* 


-.6648** 


-.8092*** 


SO 


.2913 


.2175 


.1147* 


.0181** 


100 


.3459 


.3079 


.2529 


.1987* 


200 


.3730 


.3537 


.3253 


.2967 


1000 


.3946 


.3907 


.3849 


.3789 



Note. --Asterisks refer to significance levels of unit-normal 
z tests con?Jaring r,. and r . 



• i)<.05 

£-t.01 
•*• £<.001 



Tabic 3 

Values of Resulting from One Pattern of 
Subgroup Ccntroid Differences and I'ivc Total 
Group Samjile Sizes: r^ = 0.0000 



Total group 
sample size 


Significance levels for t distribution when 
subgroup mean differences equal critical values 


£<.05 


Il<.oi 


£<.001 


I)<.0001 


Pattern 1: X2 > 


X 


1' h ' Tv 


both significant 


10 


.3993 




.5846 


. 7606* 


.8637** 


50 


.0777 




.13015 


.2038 


.2728 


100 


.0386 




.0658 


.1051 


.1438 


200 


.0193 




.0330 


.0533 


.0738 


1000 


.0038 




.0066 


.0108 


.0151 



Note. --Asterisks refer to significance levels of unit-normal 
£ tests conparing r^ and r^. 
* £<.05 

•* £<.01 
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FIG* !• Nine exaggerated bivariate plots of total proup data 
resulting from pooling heterogeneous subgroups. 
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